Towards reconstructing HDR light fields by combining 2D and 3D CNN architectures

High dynamic range imaging has recently become a technological trend, with numerous attempts to reconstruct HDR images and videos from low-dynamic-range data. The reconstruction of light field images is analogous to the reconstruction of HDR videos, within which consecutive frames are temporally coherent. For light field images, many similarities exist between the adjacent views, since they visualize the same scene from different angular perspectives. In this paper, we investigate the theoretical possibilities of combining CNN architectures utilized for HDR images and videos in order to enhance the outputs of HDR light field image reconstruction

Towards a quality metric for dense light fields.

Light fields become a popular representation of three-dimensional scenes, and there is interest in their processing, resampling, and compression. As those operations often result in loss of quality, there is a need to quantify it. In this work, we collect a new dataset of dense reference and distorted light fields as well as the corresponding quality scores which are scaled in perceptual units. The scores were acquired in a subjective experiment using an interactive light-field viewing setup. The dataset contains typical artifacts that occur in light-field processing chain due to light-field reconstruction, multi-view compression, and limitations of automultiscopic displays. We test a number of existing objective quality metrics to determine how well they can predict the quality of light fields. We find that the existing image quality metrics provide good measures of light-field quality, but require dense reference light-fields for optimal performance. For more complex tasks of comparing two distorted light fields, their performance drops significantly, which reveals the need for new, light-field-specific metrics

Too Hot to Handle: An Evaluation of the Effect of Thermal Visual Representation on User Grasping Interaction in Virtual Reality

Influence of interaction fidelity and rendering quality on perceived user experience have been largely explored in Virtual Reality (VR). However, differences in interaction choices triggered by these rendering cues have not yet been explored. We present a study analysing the effect of thermal visual cues and contextual information on 50 participants' approach to grasp and move a virtual mug. This study comprises 3 different temperature cues (baseline empty, hot and cold) and 4 contextual representations; all embedded in a VR scenario. We evaluate 2 different hand representations (abstract and human) to assess grasp metrics. Results show temperature cues influenced grasp location, with the mug handle being predominantly grasped with a smaller grasp aperture for the hot condition, while the body and top were preferred for baseline and cold conditions

3D VIDEO FORMAT CONVERSION

The main aim of this thesis work is to find/implement various methods that convert Conventional Stereoscopic 3D Video (CSV) to Multiview video (MVV). The work investigates different methods that can produce multiple views given a stereoscopic pair from a frame of a particular video sequence and continues with the process of selecting the best among investigated methods that has optimum quality and speed. In contrast to the existing algorithms, this work disregards the physical depth but instead focus on pixel value correspondence. The intermediate view generation in this work is not considered as a geometrical problem, but a morphing problem. Different morphing algorithms (mesh, field and thin plate spline morphing techniques) are considered for conversion. Performance of each morphing algorithm is in turn compared using different correspondence matching techniqes. The investigated methods aim to produce arbitrary number of novel synthesized camera views from a sparse view set. Mesh morphing algorithm is found to be a better candidate in terms of signal to noise ratio, but requires accurate correspondences at edges of an object in a particular scene and also needs more execution time to generate more number of views. A new approach to field morphing has been introduced in this thesis work, which performs better in terms of execution time and also found to produce intermediate views with reasonable signal to noise ratio. This approach is observed to bring good trade off between speed and accuracy. This conversion has an advantage it can be used as a decompression mechanism that can produce multiple views required for an Autostereoscopic 3D display from a stereoscopic left and right pair. This approach also brings the benefit of backward compatibility as present standards for CSV may be used to provide multiview 3D video to high fidelity Autostereoscopic 3D displays of the future. This work has applications in free view point television, video conferencing systems etc.,C/O SHASHIDHAR REDDY NACKSTÄVAGAN 16’C, LGH 3172, 85352 SUNDSVALL, SWEDEN. Ph: 004673478457

3D VIDEO FORMAT CONVERSION

The main aim of this thesis work is to find/implement various methods that convert Conventional Stereoscopic 3D Video (CSV) to Multiview video (MVV). The work investigates different methods that can produce multiple views given a stereoscopic pair from a frame of a particular video sequence and continues with the process of selecting the best among investigated methods that has optimum quality and speed. In contrast to the existing algorithms, this work disregards the physical depth but instead focus on pixel value correspondence. The intermediate view generation in this work is not considered as a geometrical problem, but a morphing problem. Different morphing algorithms (mesh, field and thin plate spline morphing techniques) are considered for conversion. Performance of each morphing algorithm is in turn compared using different correspondence matching techniqes. The investigated methods aim to produce arbitrary number of novel synthesized camera views from a sparse view set. Mesh morphing algorithm is found to be a better candidate in terms of signal to noise ratio, but requires accurate correspondences at edges of an object in a particular scene and also needs more execution time to generate more number of views. A new approach to field morphing has been introduced in this thesis work, which performs better in terms of execution time and also found to produce intermediate views with reasonable signal to noise ratio. This approach is observed to bring good trade off between speed and accuracy. This conversion has an advantage it can be used as a decompression mechanism that can produce multiple views required for an Autostereoscopic 3D display from a stereoscopic left and right pair. This approach also brings the benefit of backward compatibility as present standards for CSV may be used to provide multiview 3D video to high fidelity Autostereoscopic 3D displays of the future. This work has applications in free view point television, video conferencing systems etc.,C/O SHASHIDHAR REDDY NACKSTÄVAGAN 16’C, LGH 3172, 85352 SUNDSVALL, SWEDEN. Ph: 004673478457

Exploring direct 3D interaction for full horizontal parallax light field displays using leap motion controller

This paper reports on the design and evaluation of direct 3D gesture interaction with a full horizontal parallax light field display. A light field display defines a visual scene using directional light beams emitted from multiple light sources as if they are emitted from scene points. Each scene point is rendered individually resulting in more realistic and accurate 3D visualization compared to other 3D displaying technologies. We propose an interaction setup combining the visualization of objects within the Field Of View (FOV) of a light field display and their selection through freehand gesture tracked by the Leap Motion Controller. The accuracy and usefulness of the proposed interaction setup was also evaluated in a user study with test subjects. The results of the study revealed high user preference for free hand interaction with light field display as well as relatively low cognitive demand of this technique. Further, our results also revealed some limitations and adjustments of the proposed setup to be addressed in future work

История испанской литературы ХVII в.

Autostereoscopic multi view displays require multiple views of a scene to provide motion parallax. When an observer changes viewing angle different stereoscopic pairs are perceived. This allows new perspectives of the scene to be seen giving a more realistic 3D experience. However, capturing arbitrary number of views is at best cumbersome, and in some occasions impossible. Conventional stereo video (CSV) operates on two video signals captured using two cameras at two different perspectives. Generation and transmission of two views is more feasible than that of multiple views. It would be more efficient if multiple views required by an autostereoscopic display can be synthesized from these sparse set of views. This paper addresses the conversion of stereoscopic video to multiview video using the video effect morphing. Different morphing algorithms are implemented and evaluated. Contrary to traditional conversion methods, these algorithms disregard the physical depth explicitly and instead generate intermediate views using sparse sets of correspondence features and image morphing. A novel morphing algorithm is also presented that uses scale invariant feature transform (SIFT) and segmentation to construct robust correspondences features and qualitative intermediate views. All algorithms are evaluated on a subjective and objective basis and the comparison results are presented

Exploring Direct 3D Interaction for Full Horizontal Parallax Light Field Displays Using Leap Motion Controller

This paper reports on the design and evaluation of direct 3D gesture interaction with a full horizontal parallax light field display. A light field display defines a visual scene using directional light beams emitted from multiple light sources as if they are emitted from scene points. Each scene point is rendered individually resulting in more realistic and accurate 3D visualization compared to other 3D displaying technologies. We propose an interaction setup combining the visualization of objects within the Field Of View (FOV) of a light field display and their selection through freehand gesture tracked by the Leap Motion Controller. The accuracy and usefulness of the proposed interaction setup was also evaluated in a user study with test subjects. The results of the study revealed high user preference for free hand interaction with light field display as well as relatively low cognitive demand of this technique. Further, our results also revealed some limitations and adjustments of the proposed setup to be addressed in future work

Fast and efficient data reduction approach for multi-camera light field display telepresence systems

acceptedVersionPeer reviewe